REMARKS 

The above Amendments and these Remarks are in reply to the Office Action mailed March 
11,2004. 

Currently, claims 1-44 and 47-56 and 58-61 are pending in the application. Applicants have 
amended claims 1, 22, 27, 35, and 51, cancelled claims 45, 46, and 57, and added claims 59-61. 
Applicants respectfully request reconsideration of claims 1-44, 47-56, and 58 and consideration of 
claims 59-61. 

I. Written Statement of the Substance of Interview 

A statement of the substance of the interview conducted on September 2, 2004 between the 
Examiner and Applicants' representative is provided. 

The Examiner is thanked for granting Applicants' representative the opportunity to discuss 
the application and the USPTO's position relating thereto. 

Claim 1 and U.S. Patent No. 6,057,833 ^Heidmann") were discussed. 

The amendment of claim 1 to recite one of the following proposed limitations was discussed: 
(1) "defining a virtual surface relative to a camera, said defining includes representing said virtual 
surface in a three-dimensional coordinate system for said camera, selecting a reference point for said 
virtual surface relative to said camera, and positioning said virtual surface close to a portion of an 
environment likely to be enhanced in an image from said camera;" and (2) "defining a virtual surface 
at a desired position with respect to a camera." 

The Examiner agreed that the first proposed limitation would render claim 1 patentable over 
the cited art. The Examiner indicated that the second proposed limitation may render claim 1 
patentable but that further consideration would be required in order to make a final decision. 

II. Rejection of claims 1, 4-10, and 17 under 35 U.S.C. $ 102(e) 

Claims 1, 4-10, and 17 were rejected under 35 U.S.C. § 102(e) as being anticipated by U.S. 
Patent No. 6,057,833 (Heidmann). Because Heidmann does not disclose each of the limitations of 
these claims, Applicants assert that claims 1,4-10, and 17 are patentable over the cited art. 
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The Examiner indicated during the interview with Applicants' representative that the 
amendment of claim 1 to include the limitation, "defining a virtual surface relative to a camera, said 
defining includes representing said virtual surface in a three-dimensional coordinate system for said 
camera, selecting a reference point for said virtual surface relative to said camera, and positioning 
said virtual surface close to a portion of an environment to be enhanced in an image from said 
camera.," would distinguish the claim from the Heidmann reference. Accordingly, Applicants have 
amended claim 1 to recite this limitation. Thus, claim 1 as amended is believed patentable over the 
cited art. 

III. Rejection of Claims 2, 3, 11-16, 18-21, and 22-58 under 35 U.S.C § 103(a) 

Claims 2-3, 11-16, 18-21, and 22-58 were rejected under 35 U.S.C. § 103(a) as being 
unpatentable over Heidmann. Because Heidmann fails to teach or suggest each of the limitations of 
claims 2-3, 11-16, 18-21, 22-44, 47-56, and 58, Applicants assert that these claims are patentable 
over the cited art. 

A. Claims 22-34 and 49-50 

Amended independent claims 22 and 27 each recite: 

defining a virtual surface relative to a camera, said defining includes 
representing said virtual surface in a three-dimensional coordinate 
system for said camera, selecting a reference point for said virtual 
surface relative to said camera, and positioning said virtual surface 
close to a portion of an environment to be enhanced in an image from 
said camera. 

The Examiner indicated during the interview that the amendment of claim 1 to recite this 
limitation would render claim 1 patentable over Heidmann. Claims 22 and 27 have also been 
amended to include this limitation and are thus believed patentable over the cited art. Claims 23-26 
and 49-50 each ultimately depend from claim 22 and should be patentable for at the least the same 
reasons. Claims 28-34 each ultimately depend from claim 27 and should be patentable for at least 
the same reasons. 
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B. Claims 35-46 

Amended independent claim 35 recites "defining a virtual surface at a desired position with 
respect to a camera." 

Heidmann does not teach the definition of a virtual surface at a desired portion with respect 
to a camera because any virtual surface disclosed therein is defined in terms of an external physical 
3-D environment. It cannot be defined at a desired location because it represents an actual 3-D 
environment which must be accurately represented as it actually exists. 

For example, Heidmann teaches that his system has the ability to define objects as "three- 
dimensional objects located in three-dimensional space." Heidmann, col. 12, 11. 50-51. The three- 
dimensional object "can be defined as projected onto the field which is defined as a plane or the 
animation object may be defined as floating a certain known distance above the known level of the 
playing field." Id. at 11. 45-48. In this example, the field is defined as a plane surface and the objects 
can be projected onto the plane surface because the level of the playing field is known and known 
distances can be defined in terms of the known level. Accordingly, any virtual surface disclosed in 
this passage is an approximation of a real surface defined in terms of an actual external 3-D 
environment using known distances and known levels of the actual environment being captured. 
There is nothing that teaches or suggests the definition of a "virtual surface at a desired position with 
respect to a camera." as recited in claim 1. 

Heidmann contains a single section where he uses the term "virtual surface." But again, a 

careful review of this section reveals that the virtual surface refers to an approximation of an actual 

environment. Heidmann teaches that: 

graphical elements may be drawn onto a virtual surface which can be moved 
or animated in 2D or 3D, that is either scaled or slid around on the screen, or 
tumbled, viewed in perspective, warped, etc. For example a basketball play 
can be diagrammed on the surface of a court , then the court can be moved in 
perspective to view the diagram from different angles. In certain 
embodiments, the motion of the graphics is driven by measured motion of the 
camera capturing a real environment, making the graphics appear to stick to 
the real scene. Heidmann, col. 13, 11. 7-16 {emphasis added). 
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Heidmann reveals that his virtual surface approximates a real surface which in this example 
is a basketball court. Heidman says graphical elements can be drawn onto a virtual surface and 
provides the example of drawing on the surface of a basketball court. The court can then be moved 
in perspective. The virtual representation of the basketball court is the virtual surface. Accordingly, 
the virtual surface must be defined relative to the actual 3-D external environment in order to 
accurately represent the environment. There is no mention or suggestion of a virtual surface that is 
defined "at a desired position with respect to a camera. In order to approximate an actual surface 
such as a basketball court in a virtual environment, the virtual surface must be defined in terms of the 
actual surface to provide a direct correlation thereto. 

Because Heidmann fails to teach or suggest each of the limitations of claim 35, Applicants 
assert that claim 35 is patentable over the cited art. Claims 36-44 each ultimately depend from claim 
35 and should be patentable for at least the reasons set forth above. 

C. Claims 51-58 

Amended independent claim 51 recites: 

defining a virtual surface in a three-dimensional coordinate system for 
a camera, said defining includes selecting at least one dimension of 
said virtual surface, relative to said camera, such that at a least a 
portion of said virtual surface is positioned close to a portion of an 
environment to be enhanced in an image from said camera 

As set forth with respect to claim 35, Heidmann fails to teach or suggest "defining a virtual 
surface at a desired position with respect to a camera." Similarly, Heidmann makes no mention of 
defining a virtual surface "in a three-dimensional coordinate system" where defining includes 
"selecting at least one dimension of said virtual surface, relative to said camera, such that at least a 
portion of said virtual surface is positioned close to a portion of an environment to be enhanced in an 
image from said camera." 

Heidmann 's virtual surface is an approximation of an actual surface and is defined in terms of 
the external 3-D environment being captured. Because Heidmann 's surface approximates a real 
surface, its dimensions are governed by the actual dimensions of the surface being approximated. A 
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dimension of the surface cannot be selected, relative to a camera, so that it is positioned close to a 
portion of an environment to be enhanced. Given that the virtual surface in Heidmann approximates 
a real surface, its dimensions should be dictated by the actual surface in order to properly represent 
such a real surface. 

Because Heidmann fails to teach or suggest each limitation of claim 5 1, Applicants assert that 
claim 51 is patentable over the cited art. Claims 52-58 each ultimately depend from claim 51 and 
should be patentable for at least the same reasons as set forth above. 

D. Claims 2, 3, 11-16, 18-21, and 47-48 

Claims 2, 3, 11-16, 18-21, and 47-48 each ultimately depend from claim 1. As set forth 
above (with respect to claim 1 and the same limitation found in claims 22 and 27), Heidmann fails to 
teach or suggest each of the limitations of claim 1 . Accordingly, for at least the same reasons as set 
forth with respect to claims 1,22, and 27, Applicants assert that claims 2, 3, 11-16, 18-21, and 47-48 
are patentable over the cited art. 

IV. Rejection of Claim 48 under 35 U.S.C. § 112, first paragraph 

Claim 48 was rejected under 35 U.S.C. § 1 12, first paragraph, on the basis that it "is unclear 
how the disclosure teaches the claimed 'at least one of said one or more locations are not located on 
said virtual surface." 

Applicants respectfully disagree and assert that the limitations in question are properly 
supported. For example, Applicants teach that points or locations can be mathematically described 
with respect to (or connected to) the virtual surface. In this manner, the points need not actually lie 
on the virtual surface because they are mathematically described with relation thereto. 

Applicant's specification teaches that: 

In one embodiment, the points identified in step 310 are transformed 
to the three dimensional coordinate system of the virtual surface, but 
not all of the points will lie on the virtual surface - some will be on 
the virtual surface and others will be off the virtual surface but 
mathematically described with respect to (or connected to) the virtual 
surface. Various means for converting points between coordinate 
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systems are known in the art. One example of doing such a 
transformation is to use transformation matrices. Specification, p. 23, 
11. 10-17. 

Accordingly, it is respectfully submitted that claim 48 complies with § 112's, second 
paragraph written description requirement because Applicants' Specification makes clear that 
Applicants possessed, at the time of filing the present application, the claimed invention of "at least 
one of said one or more locations are not located on said virtual surface." Accordingly, withdrawal 
of the rejection under § 1 12 is respectfully requested. 

V. Newly Added Claims 

Claims 59-61 have been added. Claim 59 depends from claim 51 and claims 60-61 each 
ultimately depend from claim 35. It is respectfully submitted that these claims are patentable for at 
least the same reasons set forth above with respect to claims 35 and 51. 

VI. Conclusion 

Based on the above amendments and these remarks, reconsideration of claims 1-44, 47-56, 
and 58, and consideration of claims 59-61 is respectfully requested. 

The Examiner's prompt attention to this matter is greatly appreciated. Should further 
questions remain, the Examiner is invited to contact the undersigned attorney by telephone. 

Enclosed is a PETITION FOR EXTENSION OF TIME UNDER 37 C.F.R. § 1.136 for 
extending the time to respond up to and including today, September 9, 2004. 
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The Commissioner is authorized to charge any underpayment or credit any overpayment to 
Deposit Account No. 501 826 for any matter in connection with this response, including any fee for 
extension of time, which may be required. 



Respectfully submitted, 



Date: Stpktobv ? Z60S Bv: jC^C^ 



Vierra Magen Marcus Harmon & DeNiro LLP 

685 Market Street, Suite 540 

San Francisco, California 94105-4206 

Telephone: (415) 369-9660 

Facsimile: (415) 369-9665 
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David E. Cromer 
Reg. No. 54,768 
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